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ABSTRACT 



A software visualization tool enables a user to observe the 
inner workings of a computer program (e.g., a Java program) 
while it executes. An original program is augmented with 
calls to a shadow machine at locations where the original 
program modifies a data structure maintained by the data 
program or updates values in a value set maintained within 
the context of the data structure. 

A duplicate copy of the data structure and the value set for 
the augmented program is maintained during execution of 
the computer program by a shadow machine. A display 
engine accesses the duplicate copy of the data structure and 
the value set, and displays a visual representation of the 
duplicate copy on a display device so as to facilitate visu- 
alization of the execution of the augmented program in a 
dynamic and near real-time manner. 

70 Claims, 23 Drawing Sheets 
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METHOD AND SYSTEM FOR RUN-TIME 
VISUALIZATION OF THE FUNCTION AND 
OPERATION OF A COMPUTER PROGRAM 

This application claims priority from U.S. provisional 
application serial No. 60/139,337, entitled "METHOD AND 
APPARATUS FOR IMPLEMENTING A SOFTWARE 
VISUALIZATION TOOL", and filed Jun. 15, 1999. 

FIELD OF THE INVENTION 

The present invention relates generally to the field of 
software visualization and, more specifically, to a method 
and system for visualizing the run-time functioning and 
operations of a computer program via a visual display 
presented by a user interface. 

BACKGROUND OF THE INVENTION 

While a number of software visualization tools exist, 
some programming environments (e.g., the Java program- 
ming environment) impose restrictions on access to data 
structures maintained by program (e.g., variables). Within 
the Java programming environment, for example, local 
variables (i.e., variables contained in a method invocation 
frame) can only be accessed by code in that method and by 
an extant thread, when the relevant method currently being 
executed at the top of the extant thread's stack. Further, class 
and object variables may be declared in such a way so as to 
prevent access by code to such class and object variables 
outside the relevant class. It will be appreciated that the 
access restrictions to the structure of the computer program 
are problematic for a third party visualization tool, which 
seeks to provide a visual view of such data structures. 

By way of background to the Java program environment, 
Java is an object-oriented language. Java source code files 
(files with a java extension) are compiled into a format 
called "bytecode" (files with a .class extension), which can 
be executed by a Java interpreter. Compiled Java code can 
be run across multiple platforms by Java interpreters and 
run-time environments, known as Java Virtual Machines 
JVMs), that have been developed for a variety of platforms 
(e.g., Unix, the Macintosh Operating System, and the Win- 
dows family of operating systems). Bytecode may be con- 
verted directly into machine language instructions by a 
Just-in-Time compiler (JIT). 

FIG. 1 is a diagrammatic representation of the execution 
of a Java program. Java class files (i.e., bytecode), which 
comprises the compiled Java code, are converted to 
machine-language program code 14 by the JIT of the Java 
Virtual Machine. The program code 14 in turn dynamically 
creates and maintains a program data structure, and also 
populates the data structure 16 with a value set 18 (e.g., 
numeric, text or alphanumeric values). 

SUMMARY OF THE INVENTION 

According to a first aspect of the present invention, there 
is provided a method of visualizing functioning of a com- 
puter program. A duplicate copy of a data structure of the 
computer program is maintained during an execution (or 
run-time) of the computer program. A visual representation 
of the duplicate copy of the data structure is displayed on a 
display device so as to visualize the execution of the 
computer program. 

Other features of the present invention will be apparent 
from the accompanying drawings and from the detailed 
description which follows. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example 
and not limitation in the figures of the accompanying 
5 drawings, in which like references indicate similar elements 
and in which: 

FIG. 1 is a diagrammatic representation of a prior art 
execution of a Java program. 

FIGS. 2A and 2B are diagrammatic representations of 
10 respective exemplary embodiments of a visualization tool. 

FIG. 3 is a flow chart providing an overview of a method, 
according to an exemplary embodiment, of visualizing the 
function and operation of a computer program. 

FIG. 4 is a flow chart illustrating a method, according to 
15 an exemplary embodiment, of performing an augmenting 
operation with respect to program code. 

FIG. 5 is a block diagram illustrating the insertion of an 
exemplary shadow machine call instruction into augmented 
M program code. 

FIG. 6 provides a diagrammatic representation of an 
exemplary original data structure that may be maintained by 
a shadow machine, according to an exemplary embodiment 
of the present invention. 
25 FIG. 7 is a block diagram illustrating an exemplary 
original data structure and value set, and a duplicate data 
structure and value set as maintained by the shadow 
machine. 

FIG. 8 A is a flow chart illustrating further detail regarding 
30 a method, according to an exemplary embodiment, of aug- 
menting program code by the insertion of shadow machine 
call instructions into the program code, and the various 
operations that may be performed by the inserted shadow 
machine call instructions. 
35 FIG. 8B is a flow chart illustrating a method, according to 
an alternative exemplary embodiment, of maintaining a 
duplicate copy of a data structure and/or value set utilizing 
a modified interpreter that identifies modifying instructions 
"on the fly". 

4,0 

FIG. 9 is a diagrammatic representation of an exemplary 
object shadow corresponding to an original object that may 
maintained by the shadow machine. 

FIG. 10 is a diagrammatic representation of a display 
45 engine, according to an exemplary embodiment of the 
present invention. 

FIGS. 11A-UD illustrate exemplary panels, or interfaces, 
that may be presented by a launcher component of the 
display engine. 

50 FIGS. 12A-12F illustrate further exemplary interfaces 

that may be presented within the context of a display 

window generated by the display engine. 

FIG. 13 is a diagrammatic representation of a machine, in 

the exemplary form of a computer system, within which a 
55 set of instructions for causing a machine to perform any of 

the exemplary methodologies of the present invention may 

be executed. 

DETAILED DESCRIPTION 

60 

A method and system for visualizing the functioning of a 
computer program are described. In the following 
description, for purposes of explanation, numerous specific 
details are set forth in order to provide a thorough under- 
65 standing of the present invention. It will be evident, 
however, to one skilled in the art that the present invention 
may be practiced without these specific details. 
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Overview 

During the course of the execution of a computer 
program, for example written in an object-oriented 
language, the computer program may create new objects, 
destroy existing objects, change references between objects, 
and modify the values of variables contained within objects. 
The present invention proposes a method and system that 
presents a visualization of operations and functioning of a 
computer program that modifies the state of data. The 
invention, in one embodiment, proposes presenting and 
updating a corresponding visual representation in a display 
window so that a user may observe the state of the computer 
program as it evolves and is modified any time during 
execution of the computer program. 

The present invention is described within the context of 
the Java programming environment, and utilizing the Java 
programming language, as developed by Sun Microsystems. 
Nonetheless, it will be appreciated that the teachings of the 
present invention are not limited to a Java environment, and 
may be utilized to provide access to the data structure and 
value set of any computer program wherein the relevant 
program environment poses restrictions on access to such 
data structures and/or such value sets. A visualization pro- 
gram that comprises one embodiment of the present 
invention, is described below as being implemented in Java 
in order to make it platform-independent. Again, it will be 
appreciated that the visualization program may be written in 
any programming language, and is not limited to the Java 
programming language. 

FIG. 2A is a diagrammatic representation of an exemplary 
embodiment of a visualization tool 20 that embodies a 
method and system for visualizing the functioning and 
operations of the computer program. The visualization tool 
20 is shown to include three major sub-systems, namely an 
augmenter 22, in the exemplary form of an augmenting class 
loader, a shadow machine 24 that maintains a duplicate copy 
27 of the data structure and value set 28 of an augmented 
program code 30, and a display engine 26 which provides a 
visual representation of the duplicate copy of the data 
structure and value set maintained by the shadow machine 
24. Further details regarding each of the subsystems of the 
visualization tool 20 will be discussed in further detail 
below. FIG. 2B is a diagrammatic representation of an 
alternative embodiment of the visualization tool 20. 

FIG. 3 is a flow chart providing an overview of a method 
40, according to an exemplary embodiment, of visualizing 
the functioning and operation of the computer program. The 
various operations of the method 40 are performed by the 
respective sub-systems of the visualization tool 20. 
Specifically, at block 42, the augmenter 22 augments a 
program code to generate the augmented program code 30 to 
include calls to the shadow machine 24. Each call to the 
shadow machine 24 includes a data structure identifier 
and/or an update value that the shadow machine 24 utilizes 
to maintain the duplicate copy 27 of the data structure and/or 
the value set. 

At block 44, the shadow machine 24 maintains the 
duplicate copy 27 of the data structure and value set utilizing 
the augmented program code 30, during execution of such 
augmented program code 30. 

At block 46, the display engine 26 displays a visual 
representation of the duplicate copy 27 of the data structure 
and/or value set as maintained by the shadow machine 24 on 
a user display. The display of the visual representation may, 
in one embodiment, be presented in a dynamic manner 
during run-time of the augmented program code 30 so as to 
provide a dynamic, near real-time view of the internal 
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operations and functions performed by the augumented 
program code 30. 
Augmenter (22) 

One embodiment of the present invention requires that 

s original program code (e.g., bytecode) be augmented prior 
to execution to produce instructions that issue calls to the 
shadow machine 24 so as to maintain a synchronized state 
between original program data structure 16 and the value set 
18, and a duplicate copy 27 of the original data structure and 

10 value set, as maintained by the shadow machine 24. 

FIG. 4 is a flow chart illustrating a method 42, according 
to an exemplary embodiment, of performing the augmenting 
operation described above at block 42 with reference to FIG. 
3. The method 42 commences at block 50, with the aug- 

15 menter 22 examining a sequence of code (e.g., the class files 
(bytecode) 12). In one embodiment of the present invention, 
the augmenter 22 may be implemented as the augmenting 
class loader illustrated in FIG. 2 that augments the class files 
12 (on the fly) as they are loaded for execution. To this end, 

20 the Java programming environment provides a mechanism 
whereby a customized class loader (i.e., the augmenting 
class loader) may be defined and used to load classes. The 
augmenting class loader may locate and read classes in the 
same manner as a default class loader, and augment class 

25 files before communicating them to a Java Virtual Machine 
for loading and linking. 

In the alternative embodiment shown in FIG. 2B, the 
augmenter 22 is an independent program that reads the class 
files 12 and produces augmented class files 13 that may be 

30 loaded by a default class loader 15. 

At block 52, the augmenter identifies update instructions 
within the class files 12 that potentially modify or update the 
data structure or value set maintained by the augmented 
program code 30. For example, the augmenter 22 may 

35 identify update instructions that (1) modify variables, (2) 
invoke methods, or (3) return from methods. 

At block 54, having identified such update instructions, 
the augmenter 22 then proceeds to insert shadow machine 
call instructions 56 (e.g., additional instructions in the form 

40 of bytecode that call the shadow machine 24) into the 
original bytecode. These additional bytecodes (i.e., 
instructions) are associated with update instructions, and 
pass information that specifies details of the operation 
performed by the relevant update instructions. For example, 

45 where an update instruction modifies a value for a variable, 
the inserted augmenting instructions (i.e., shadow machine 
call) may communicate details concerning the relevant 
object, a field, and an update value to the shadow machine 
24. The shadow machine 24 may then utilize such inform a- 

50 tion to update a corresponding shadow field in a shadow 
object so as to maintain a synchronous stale between the 
original program data structure and value set and the dupli- 
cate copy 27 thereof. Further details regarding the execution 
of such augmenting instructions are provided below. 

55 FIG. 5 is a block diagram illustrating unmodified program 
code in the form of class files 12, including instructions that 
modify a variable, invoke a method and return from a 
method. FIG. 5 further illustrates how shadow machine call 
instructions 56 are inserted into augmented program code 13 

60 (uncompiled) or 30 (compiled) so as to communicate appro- 
priate update information to the shadow machine 24. 
Shadow Machine (24) 

In order to allow the visualization tool 20 to determine the 
value of any variable, or the data structure, of an executing 

65 program, the shadow machine 24 maintains the duplicate 
copy 27 of such a data structure and/or value set. To enable 
the shadow machine 24 to perform this function, the aug- 
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mented program code 30, as described above, communicates 
update information to the shadow machine 24. The shadow 
machine 24 includes a copy of the entire data structure of an 
executing program, as well as a copy of the value set stored 
in the context of such a data structure. 5 

It may be useful to consider an exemplary data structure 
maintained by a computer program. Taking a Java program 
as an example, an executing Java program typically main- 
tains its state explicitly within variables (named containers, 
each of which holds a value that the Java program can 10 
modify) and implicitly in threads (entities that execute Java 
instructions, one after the other). 

FIG. 6 provides an exemplary representation of at least a 
portion of an exemplary original data structure 16 that may 
be mirrored (or shadowed) by the shadow machine 24. is 
Specifically, FIG. 6 shows a collection of threads 60 and a 
collection of variables 62, each thread and variable having 
a corresponding shadow maintained by the shadow machine 
24. 

Each thread of the Java program may maintain a stack of 20 
method invocation frames 64, each frame corresponding to 
an unfinished invocation of a method. Each frame may 
contain a collection 66 of variables. As illustrated, variables 
can also be contained in classes (related to variables and 
methods), objects (instantiations of classes), and arrays 25 
(indexed collections of variables of the same type). 

FIG. 7 is a block diagram illustrating the original data 
structure and/or value set maintained for the executing 
program, as well as the duplicate data structure and value set 
maintained by the shadow machine 24. Specifically, the 30 
duplicate data structure is shown to include thread shadows 
70, frame shadows 72, class shadows 74, object shadows 76 
and array shadows 78. Further details after each of these 
shadow data structures will provided below with reference 
to FIG. 8. 35 

FIG. 8A is a flow chart illustrating a method 44, according 
to an exemplary embodiment, of maintaining the duplicate 
copy 27 of the data structure and/or value set by the 
generation of the augmented program code 30. The method 
44 provides further details regarding exemplary method 44 40 
of performing an augmenting operation, as described with 
FIG. 3. Specifically, FIG. 8A illustrates operation that may 
be invoked by an inserted shadow machine call instruction 
56. 

The method 44 commences at block 80 with the augment- 45 
ing of the program code 30. At decision block 82, update 
instructions that modify a field of an object are identified. 
Following a positive determination at decision block 82, the 
method 44 proceeds to block 83, where the augmenter 22 
inserts a "modify" shadow machine call instruction 56 into 50 
the program code, the inserted instruction 56 being associ- 
ated with the update instruction at decision block 82. The 
various operations that are preformed responsive to the 
"modify" shadow machine call instruction 56 are illustrated 
in blocks 84 through 92. It will be appreciated that these 55 
operations are only performed at run-time of the augmented 
program code 30, and not during actual augmenting of the 
original program code, as has been described with reference 
to FIG. 8A. Further details regarding each of the operations 
performed responsive to an actual execution of the "modify" 60 
shadow machine call instruction 56 will now be described. 

At block 84, a shadow machine call is issued from the 
augmented program code 30, required by an appropriate 
shadow machine call instruction 56. The shadow machine 
call specifies an object of the original data structure 16 to 65 
which the update applies, a field of the specified object to 
which the updated applies, and an update value. As 



described above, each object of the original data structure 16 
has a corresponding object shadow 76 maintained by the 
shadow machine 24. Each object shadow 76 in rum main- 
tains copies of all the fields that make up the original object. 

At block 86, the shadow machine 24 maps the object 
identified by the call to a shadow object. To this end, the 
shadow machine 24 maintains a mapping (not shown) 
between objects and the corresponding object shadows so as 
to enable the shadow machine 24 to locate an object shadow 
corresponding to a specified object. 

At block 88, the shadow machine 24, from the identified 
object shadow 76, identifies a corresponding class shadow 
74. 

At block 90, the shadow machine 24, from the class 
shadow 74, determines the location of a shadow copy of the 
specified field within an array shadow 78. 

At block 92, the shadow machine 24 then assigns the data 
value to the shadow copy of the specified field within the 
identified array shadow 78. 

It should be noted that each object shadow 76 maintains 
the values of all fields of the corresponding original object. 
Accordingly, each object shadow 76 is able to accommodate 
the various quantity and types of fields specified in any one 
of multiple different classes. To achieve this in a type-safe 
manner, each object shadow 76 contains an array for each 
fundamental title (e.g., int, long, float, double, etc.). Each 
field of the corresponding class is mapped to a particular 
element in the array of the corresponding class. Values in the 
array shadows 78 are updated as their associated fields in the 
original objects of the original data structure 16 are changed. 
FIG. 9 provides a diagrammatic representation of an exem- 
plary object shadow 76 corresponding to an original object 
containing four integer and two double variables. 

As stated above, object shadows are required to be able to 
deal with the various bits of data associated with all of the 
various classes provided within the original data structure 16 
that may be loaded and utilized by the augmented program 
30. Each particular class defines a fixed set of object 
variables. Accordingly, within the visualization tool 20, each 
object shadow 76 has a corresponding class shadow 74, 
which maintains the sets of fields and mappings for them to 
the associated type indices. For example, consider the fol- 
lowing class definition fragment: 



Class Foo { 

private int count; 
private double x; 
private double y; 
private int index; 
private Foo next; 



} 



The class shadow 74 corresponding to an original class 
Foo will maintain the following mappings: 



0 


"count": 


int 


0 (index in int array) 


1 


V 


double 


0 (index in double array) 


2 




double 


1 (index in double array) 


3 


"index" 


int 


1 (index in int array) 


4 


"next" 


object 


0 (index in object array) 



Accordingly, each object shadow 76 maintains a reference 
75, to a corresponding class shadow 74, as illustrated in FIG. 
7. Thus, for example, when the augmented program code 30 
sends a value to a variable in an object of a task, the 
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augmented program code 30 calls the shadow machine 24 
specifying the object, the field index and the update value. 
A value is then assigned to a shadow variable in the manner 
described above with reference to blocks 84-92. 

Returning to method 44 illustrated in FIG. 8, at decision 
block 941 should an instruction that invokes a method entry 
be detected, the method 44 proceeds to block 96. At block 
96, the augmenter proceeds to insert a "method entry" 
shadow machine call instruction 56. During actual execution 
of the augmented program code 30, and responsive to the 
"method entry" shadow machine call instruction 56, the 
shadow machine 24 allocates a method frame shadow 72 
and appends the allocated method frame shadow 72 to a list 
of such frame shadows for a current thread being executed 
by the augmented program code 30. 

At decision block 98, a determination is made as to 
whether an instruction within the augmented program code 
30 is detected that invokes a method return. Following a 
positive determination at decision block 98, the method 44 



interpreter, and thus not deployable across various plat- 
forms. However, an advantage of such an approach is that 
there is no requirement to pre-process (e.g., augment) the 
class files 13 as described above. 
S FIG. 8B is a flow chart illustrating a method 224, accord- 
ing to an exemplary embodiment of the present invention, 
illustrating operations that may be performed by a modified 
interpreter, such as that discussed above, that may at run- 
time identify instructions "on the fly" and issue shadow 
10 machine calls responsive to such instructions. Instructions 
that may be of interest, and invoke such shadow machine 
calls, are identified at decision blocks 282, 294 and 298. The 
operations that are performed responsive to such shadow 
machine calls by a modified interpreter are substantially the 
15 same as those discussed with reference to FIG. 8A and do 
not require further explanation. 
Display Engine (26) 

Referring back to FIG. 2A, the display engine 26 com- 
prises part of the visualization tool 20 and is called by the 



proceeds to block 100. At block 100, the augmenter 22 20 shadow machine 24 to update the contents of a display 



proceeds to insert a "method exit" shadow machine call 
instruction 56. Upon actual execution of the augmented 
code, and responsive to the "method exit" shadow machine 
call instruction 56, the shadow machine 24 removes a top 
method frame shadow from a list for a current thread. 

By way of further information regarding the operations 
performed responsive to the instructions inserted at blocks 
96 and 100, the augmented program code 30 calls the 
shadow machine 24 whenever a method is entered, and 



window 110 whenever there is an update to a variable of the 
duplicate copy of the data structure. The display engine 26 
is also responsive to input from the user via an input device, 
such as a cursor control device (e.g., a mouse) or a keyboard. 
25 The display of all information pertaining to the dynamic 
state of an executing computer program may prove over- 
whelming to a user and accordingly be of limited use. The 
display engine 26, according to one embodiment, includes 
facilities that allow a user to specify a subset of the duplicate 



whenever a return is made from a previously entered 30 copy 27 of the data structure and/or value set that the user 

method. This allows the visualization tool 20 to keep track wishes to view. Specifically, the present invention contem- 

of the backtrace of method calls for each thread of execution plates that a user may, via interface provided by the display 

by the augmented program code 30. As describe above, each engine 26, indicate a particular subset of the duplicate copy 

thread shadow 70, which minors a thread in the original data 27 of the data structure (e.g., particular classes) that the user 

structure 16, comprises a list of method frame shadows 72, 35 is interested in viewing. In addition, for each data item (e.g., 



one corresponding to each active method which has been 
called but not yet returned. Whenever a method is entered by 
the augmented program code 30, the shadow machine 24 is 
called. The shadow machine 24 allocates another method 
frame shadow 72 to the list corresponding to the current 40 
thread. Upon return, a top method frame shadow 72 is 
removed from the list. 

Each method frame shadow 72 corresponds to active 
implementation of a method by the augmented program 



class or object) that is displayed, the user may specify which 
contained variables, and associated values, are to be shown 
and which are to be hidden. In this way a user is presented 
with the option of filtering out extraneous data. 

The display engine 26 also allows a user to examine 
displayed data in multiple views, as will be described in 
further detail below. Specifically, a user may zoom in to 
view more detail, and zoom out to see an overall view of a 
data structure. Scrolling and panning of views is also sup- 



code 30, and may contain multiple local variables of various 45 ported. A user may also control the speed at which a program 



types. As with an object shadow 76, each method frame 
shadow 72 may contain one array per fundamental type to 
hold copies of current values of local variables. Whenever 
the augmented program code 30 sets a value to a local 
variable, the shadow machine 24 is called and the corre- 
sponding array element in the appropriate type array is 
updated. Each class shadow 74 contains method frame 
shadows 72 for each method defined therein, and each of 
these in turn contains the mappings for local variables to 
type array indices. 

In an alternative embodiment of the present invention, an 
interpreter may be modified such that, at run-time, inter- 
preter may identify update instructions "on the fly" as they 
are executed by the interpreter, and issue a shadow machine 



executes, and can pause or single step the execution of the 
augmented program code 30. 

FIG. 10 is a diagrammatic representation of a display 
engine 26, according to an exemplary embodiment. The 
50 display engine 26 is shown to include a launcher 102 and a 
display window 110. The launcher 102 facilitates the execu- 
tion of the augmented program code 30 under a visualization 
environment provided by the visualization tool 20. The 
launcher 102 includes an applications panel 104, a class path 
55 panel 106 and a filter panel 108, which will be described in 
further detail below. At a high level, the applications panel 
104 allows a user to identify a program to be executed in 
conjunction with the visualization tool 20, the class path 



panel 106 allows a user to include appropriate class files 

call directly to the shadow machine 24. In this way, the need 60 (e.g., jar, zip, etc.) for display, and the filter panel 108 

to generate augmented program code 30 is obviated. enables a user to select for display any combination of 

However, in order to generate such a modified interpreter, packages or classes. 

access to interpreter source code is required, this access not The display engine 26, in one embodiment, dynamically 

always being available to the developer of a visualization reveals the inner workings of a computer program (e.g., a 

tool 20. Further, the modification of an interpreter (which 65 Java application or applet) at run-time providing visual 

will typically be platform specific to its implementation) representations of data structures that are displayed "on the 

requires that the visualization tool 20 be tied to the modified fly" as a new real-time manifestation. The display engine 26 
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furthermore displays associations of objects, and provides 
information regarding references between such instances by 
employing a node- link representation. At a high level, one 
embodiment of a display engine 26 may provide a run-time 
display of class instantiation, the selected display of indi- 
vidual packages, classes and fields, and the dynamic display 
of data structures including class association, data variable 
updating and reference variable modification. 

Dealing first with the applications panel 104, FIGS. 11A 
and 11 B illustrate respective exemplary embodiments of the 
applications panel 104. The embodiment shown in FIG. 11A 
is utilized when the program code to be executed comprises 
an applet, whereas the embodiment shown in FIG. 11B is 
utilized when the executed program code comprises an 
application. The embodiment of the applications panel 104 
shown in FIG. HA includes an HTML file field 120 that 
contains the name of the file that specifies the applet to be 
executed (or run). The contents of this field are inputted 
manually or by clicking on the "browse button" to invoke a 
browser by which a user may navigate to a directory 
containing the file. 

A Java Virtual Machine (JVM) options field 124 allows a 
user to enter a JVM-specific option to assist the execution of 
the applet under the visualization tool 20 utilizing a local 
system JVM. For example, a user may enter an option 
instructing the JVM to allocate more memory to the visu- 
alization tool 20. 

Turning to the embodiment of the applications panel 104 
shown in FIG. 11B, this window includes an application 
field 126 specifying a fully qualified class name for the 
application to be executed. An arguments field 128 specifies 
the location of a supporting data file, and/or other param- 
eters. 

FIG. 11C provides an exemplary representation of the 
class path panel 106 by which a user may specify all class 
fields that the visualization tool 20 requires to successfully 
load an applet or application. Specifically, within the class 
path panel 106, a user may compose an ordered list of 
directories (containers), as well as jar and .zip files, that 
contain appropriate class files 12. 

An upper portion 130 of the class path panel 106 provides 
a directory tree for navigation purposes. Directories and files 
are listed in an active classpass list, which is maintained 
within the bottom portion 132 of the classpath panel 106. 
Directories and files in the active classpath list may be listed 
from top to bottom in the order in which they are to be 
loaded by the visualization tool 20. A user is presented with 
the option of rearranging the directories and files in the 
active classpath list utilizing the arrow buttons indicated at 
134. 

FIG. 11D provides an exemplary representation of the 
filter panel 108 by which a user may filter out classes, for 
example, that do not contribute towards a data structure that 
the user wishes to observe. 

Specifically, the filter panel 108 may display a package/ 
directory tree through which a user may enable/disable 
individual classes or entire packages for presentation by the 
display engine 26. As indicated, in one embodiment, pack- 
ages are identified by "turner" icons 136 and display enable r 
check boxes 138. A package directory is open and displayed 
when an associated turner icon 136 is pointed downward. 
Each package may contain any number of classes, and 
subpackages are displayed in an indented hierarchical form. 
Classes are identified by display enabler check boxes 138 to 
the left of the names, without turner icons 136. 

An entire package, or a single class within a package, may 
be enabled or disabled by an appropriate selection within an 
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appropriate display enabler check box 138. As illustrated, 
the check boxes 138 provide three functional states for a 
particular package, namely an "all disabled*' state, indicated 
by an empty check box 138, an "all enabled" state indicated 

5 by a crossed check box, and a "partially enabled" state 
indicated by a single traversing line through the check box 
138, as illustrated at 140. At a package level, a "all disabled" 
indication indicates all classes within the specified package 
are disabled. An "all enabled" indication indicates that all 

10 classes within the specified package are enabled for viewing 
by the display engine 24. A "partially enabled" indication 
indicates that only a specified number of classes within the 
respective package are enabled. 
At a class level, an "all enabled" indication indicates that 

15 the specified class is enabled for viewing utilizing display 
engine 26, while an "all disabled" indication indicates that 
the specified class is disabled for viewing. 

FIG. 12A provides an exemplary representation of a 
display window 110. The display window 110 is shown, in 

20 one embodiment, to be divided into two distinct areas, in the 
form of panes. Specifically, an overview pane 150 presents 
an overview of at least a portion of the duplicate copy 27 of 
the data structure maintained by the shadow machine 24. A 
detail pane 152 provides a close view (or zoomed-in view) 

25 of a portion of the data structure shown in the overview pane 
150. Specifically, the detail pane 152 displays at least a 
portion of the contents of each object, and includes arrows 
corresponding to reference variables. 
The overview pane 150 includes a gray rectangle, con- 

30 veniently termed a "panner" 154. In one embodiment, the 
size and shape of the panner 154 corresponds exactly to the 
visible region displayed in the detail pane 152. By dragging 
the panner 154 in the overview pane 150, a user may scroll 
the visible display in the detail pane 152. 

35 In the overview and detail panes 150 and 152, nodes may 
represent instances and arrows (links) may represent object 
references. Text fields within the nodes reflect the names and 
current values of, for example, object data variables. 
In addition to utilizing the panner 154 to dictate the 

40 contents of the detail pane 152, a user may select a node in 
the overview pane 150, and the detail pane 152 will scroll to 
reveal the details of the selected node. 

The display window 110 further includes a properties bar 
156 in which the properties, name and current value for a 

45 selected data field of a relevant data structure represented by 
a node displayed within the detail pane 152. This feature is 
useful when the field name is too long to be fully displayed 
within a node, or if the variable type is unknown. FIG. 12B 
illustrates a selected variable name located within a selected 

50 node 158 that is fully displayed within the properties bar 
156. 

The display window 110 further includes a tool bar 160 
that includes a collection of components. Specifically, zoom- 
in and zoom-out buttons 162 and 164 allow a user incre- 

55 mentally to magnify, or demagnify, the window image 
within the overview pane 150. A "pause" button 166 allows 
a user, during run-time of an augmented program code 30, 
to pause the construction and maintenance of the duplicate 
copy 27 of the original data structure. To resume real-time 

60 rendering of the display, a "resume" button 168 is presented 
for user selection. 

A collection of split-screen buttons 170, 172 and 174 are 
also presented within the tool bar 160. Specifically, a left/ 
right split-screen button 170 allows the overview and detail 

65 panes 150 and 152 to be divided vertically, as shown in FIG. 
12C. A top/bottom split-screen button 172 provides a hori- 
zontal division between the overview and detail panes 150 
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and 152. Finally, a single pane button 174 allows a user to 
achieve a larger view of an entire structure that may be too 
large to fit into a split-screen display. An example of such a 
single pane display is provided in FIG. 12D. In the single 
pane display, the display functions in the same manner as the 5 
overview window 150. 

Finally, the toolbar 160 includes respective "horizontal- 
build" and "vertical-build" buttons 176 and 178. User selec- 
tion of the horizontal-build button 176 causes a link-node 
representation of the duplicate copy 27 of the data structure 10 
to be horizontally rendered, while user selection of the 
vertical-build button 178 causes a link-node representation 
to be vertically rendered. It is useful, for example, to render 
a tree structure vertically, whereas a hash table may be better 
viewed utilizing a horizontal orientation. FIG. 12E provides 15 
an example of a vertical button node presentation whereas 
FIG. 12F provides an example of a horizontal built repre- 
sentation. 
Computer System 

FIG. 13 shows a diagrammatic representation of machine 20 
in the exemplary form of a computer system 200 within 
which a set of instructions, for causing the machine to 
perform any one of the methodologies discussed above, may 
be executed. In alternative embodiments, the machine may 
comprise a network router, a network switch, a network 25 
bridge, Personal Digital Assistant (PDA), a cellular 
telephone, a web appliance or any machine capable of 
executing a sequence of instructions that specify actions to 
be taken by that machine. 

The computer system 320 includes a processor 202, a 30 
main memory 204 and a static memory 206, which com- 
municate with each other via a bus 208. The computer 
system 200 may further include a video display unit 210 
(e.g., a liquid crystal display (LCD) or a cathode ray tube 
(CRT)). The computer system 200 also includes an alpha- 35 
numeric input device 212 (e.g. a keyboard), a cursor control 
device 214 (e.g. a mouse), a disk drive unit 216, a signal 
generation device 218 (e.g. a speaker) and a network inter- 
face device 220. 

The disk drive unit 216 includes a machine-readable 40 
medium 222 on which is stored a set of instructions (i.e., 
software) 224 embodying any one, or all, of the method- 
ologies described above. The software 224 is also shown to 
reside, completely or at least partially, within the main 
memory 204 and/or within the processor 202. The software 45 
224 may further be transmitted or received via the network 
interface device 220. For the purposes of this specification, 
the term "machine-readable medium" shall be taken to 
include any medium which is capable of storing or encoding 
a sequence of instructions for execution by the machine and 50 
that cause the machine to perform any one of the method- 
ologies of the present invention. The term " machine - 
readable medium" shall accordingly be taken to included, 
but not be limited to, solid-state memories, optical and 
magnetic disks, and carrier wave signals. 55 

Thus, a method and system for visualizing the functioning 
of a computer program have been described. Although the 
present invention has been described with reference to 
specific exemplary embodiments, it will be evident that 
various modifications and changes may be made to these 60 
embodiments without departing from the broader spirit and 
scope of the invention. Accordingly, the specification and 
drawings are to be regarded in an illustrative rather than a 
restrictive sense. 

What is claimed is: 65 

1. A method of visualizing execution of a computer 
program, the method including: 
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maintaining a duplicate copy of a data structure of a 
computer program during execution of the computer 
program; and 

displaying a visual representation of the duplicate copy of 
the data structure on a display device so as to provide 
a visualization of the execution of the computer pro- 
gram. 

2. The method of claim 1 including maintaining a dupli- 
cate copy of a value set of the computer program during 
execution of the computer program and displaying a visual 
representation of the duplicate copy of the value set on the 
display device. 

3. The method of claim 2 wherein the maintaining 
includes issuing a call from the computer program, during 
execution thereof, to the duplicate copy of the value set. 

4. The method of claim 3 wherein the call communicates 
a variable identifier and an update value, wherein the update 
value is attributed to a duplicate variable of the duplicate 
copy of the data structure. 

5. The method of claim 2 including augmenting the 
computer program with call instructions to facilitate the 
maintenance of the duplicate copies of the data structure and 
the value set of the computer program. 

6. The method of claim 5 including identifying a first 
instruction within the computer program that modifies a first 
variable value and augmenting the computer program with 
a first call instruction that communicates a variable identifier 
for the first variable and an update value for the first variable. 

I. The method of claim 6 including identifying a duplicate 
first variable corresponding to the first variable and updating 
a value for the duplicate first variable utilizing the update 
value. 

8. The method of claim 6 including identifying a second 
instruction within the computer program that causes entry of 
a method by the computer program and augmenting the 
computer program with a second call instruction that adds a 
duplicate method frame to a list within the duplicate copy of 
the data structure. 

9. The method of claim 8 including identifying a third 
instruction within the computer program that causes exit 
from a method by the computer program and augmenting the 
computer program with a third call instruction that removes 
a duplicate method frame from a list within the duplicate 
copy of the data structure. 

10. The method of claim 2 wherein the display of the 
visual representation includes displaying a visual represen- 
tation of the duplicate copy of the value set. 

II. The method of claim 10 wherein the value set includes 
variable values and the visual representation provides a 
run-time display of variable value updating as reflected in 
the duplicate copy of the value set. 

12. The method of claim 2 wherein the visual represen- 
tation is dynamically varied to reflect modifications to the 
duplicate copy of the value set during run -time of the 
computer program. 

13. The method of claim 1 wherein the display of the 
visual representation includes displaying a node-link repre- 
sentation of the duplicate copy of the data structure. 

14. The method of claim 13 wherein a first node of the 
link-node representation represents a class of the data struc- 
ture as reflected in the duplicate copy thereof. 

15. The method of claim 13 wherein a node of the 
link-node representation represents an object of the data 
structure, the object comprising an instantiation of a class of 
the data structure as reflected in the duplicate copy thereof. 

16. The method of claim 13 wherein a first link of the 
link-node representation represents an object reference. 
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17. The method of claim 13 including, responsive to a a display engine to display a visual representation of the 
user selection, displaying the link-node representation in a duplicate copy of the data structure on a display device 
horizontal orientation. so as to provide a visualization of the operation of the 

18. The method of claim 13 including, responsive to a computer program. 

user selection, displaying the link node representation in a s 35. The system of claim 34 wherein the shadow structure 

vertical orientation. ... is to maintain a duplicate copy of a value set of the computer 

19. The method of claim 1 wherein the maintaining of the program during execution of the computer program and the 
duplicate copy of the data structure includes issuing a call ^ pUy enginc ^ to a visual representation of the 
from the computer program, during the execution thereof, to duplic ate copy of the value set on the display device. 

the duplicate copy of the data structure. 3fi The m of daim 35 whcrem ^ shadow structurc 

20. The method of claim 19 wherein the call communi- receives a J u from the computer program, during execution 
cates entry to a method by the computer program, responsive . ,. c 4 . . .r r lu r 

u- u a v • *u a e S • aaZa * n \ Uc* f nr o thereof, to modify the duplicate copy of the value set. 

to which a duphcate method frame is added to a list for a *- ™ . r 1 L • iL n 

current thread of the computer program. 37 ' ™ e ?Y**™ of .^ laim 3 * whe f D th * cal1 

21. The method of claim 19 wherein the call communi- cates a vanable identifier and an update value, wherein the 
cates exit of a method by the computer program, responsive ™ update value is attributed to a duplicate vanable of the 
to which a duplicate method frame is removed from a list for duphcate copy of the data structure by the shadow structure, 
a current thread of the computer program. 38. The system of claim 35 including an a\igmenter to 

22. The method of claim 21 wherein the duplicate method augment the computer program with call instructions to 
frame is removed from the top of the list. facilitate the maintenance of the duplicate copies of the data 

23. The method of claim 1 wherein the duplicate copy of 20 structure and the value set of the computer program. 

the data structure includes at least one of a group including 39. The system of claim 38 wherein the augmenter 

duplicate thread, frame, class, object and array structures. identifies a first instruction within the computer program that 

24. The method of claim 1 wherein the computer program modifies a value of a first variable and augments the corn- 
comprises Java instructions executable by a Java Virtual puter program with a first call instruction that communicates 
Machine. 25 a variable identifier for the first variable and an update value 

25. The method of claim 1 wherein a programming f or me ^ { variable. 

environment of the computer program places restrictions on ^ ^ system 0 f clami 39 wherein the augmenter 

access to the data structure and a value set of the computer identifies a duplicate first variable corresponding to the first 

program dunng execution thereof t variable and updates a value for the duphcate first variable 

26 The method of claim 1 wherein the display of the 3Q {h& ^ y ^ 

visual representation . includes displaying a graphical repre- 39 whereiD nter 

sentation of the duphcate copy of the data structure. 7 , . . . =^ 

27. The method of claim 26 wherein the graphical rep- ^ cnUfics a second instrucUcm within the computer program 
resentation is dynamically varied to reflect modifications to lhal causes ^ of a method b ? the computer program and 
the duplicate copy of the data structure during a run-time of augments the computer program with a second call instruc- 
the computer program. 35 ^ on ^ acIds a du P licate method frame to a list within the 

28. The method of claim 27 wherein the data structure of duphcate coy of the data structure. 

the computer includes classes and objects instantiated from 42. The system of claim 41 wherein the augmenter 

the classes, and the graphical representation provides a identifies a third instruction within the computer program 

run-time display of class instantiations as reflected in the that causes exit from a method by the computer program and 

duplicate copy of the data structure. 40 augments the computer program with a third call instruction 

29. The method of claim 27 wherein the graphical rep- that removes a duplicate method frame from a list within the 
resentation provides a run-time display of reference variable duplicate copy of the data structure. 

modifications as reflected in the duplicate copy of the data 43. The system of claim 35 wherein the display engine 

structure. dynamically varies the visual representation to reflect modi- 

30. The method of claim 1 wherein the display of the 45 fications to the duplicate copy of the value set during 
visual representation includes displaying a zoomed-out view run-time of the computer program. 

of the data structure, as represented by the duplicate copy, in 44. The system of claim 1 wherein the value set includes 

a first graphically distinct area and displaying a zoom-in variable values and the display engine displays the visual 

view of the data structure, as represented by the duplicate representation to provide a run-time display of variable 

copy, in a second graphically distinct area. 50 value updating as reflected in the duphcate copy of the value 

31. The method of claim 30 wherein a data structure se t. 

representation presented only in the zoomed-in view pre- 45. The system of claim 34 wherein the shadow structure 

sents variable and value information. receives a call from the computer program, during the 

32. The method of claim 30 including defining a pan view execution thereof, to modify the duplicate copy of the data 
region within the first graphically distinct area that is user 55 structure. 

beatable with reference to the zoomed-out view of the data 46. The system of claim 45 wherein the call received by 

structure, a portion of the duplicate of the data structure the shadow structure is issued as a result of execution of a 

displayed within the pan view region being displayed within call instruction included within the computer program, 

the second graphically distinct area. 47. The system of claim 45 wherein the call received by 

33. The method of claim 30 wherein both the first and 60 the shadow structure is issued as a result of a detection of a 
second graphically distinct areas can be zoomed into and out predetermined instruction type in the computer program by 
of responsive to a user selection. an interpreter. 

34. A system to visualize operation of a computer 48. The system of claim 45 wherein the call received by 
program, the system including: the shadow structure is issued as a result of execution of a 

a shadow structure to maintain a duplicate copy of a data 65 call instruction included within the computer program, 

structure of a computer program during execution of 49. The system of claim 45 wherein the call communi- 

the computer program; and cates entry to a method by the computer program, responsive 
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to which a duplicate method frame is added to a list for a 
current thread of the computer program by the shadow 
structure. 

50. The system of claim 45 wherein the call communi- 
cates exit of a method by the computer program, responsive 
to which a duplicate method frame is removed from a list for 
a current thread of the computer program by the shadow 
structure. 

51. The system of claim 50 wherein the duplicate method 
frame is removed from the top of the list. 

52. The system of claim 34 wherein the duplicate copy of 
the data structure includes at least one of a group including 
duplicate thread, frame, class, object and array structures. 

53. The system of claim 34 wherein the computer pro- 
gram comprises Java instructions executable by a Java 
Virtual Machine. 

54. The system of claim 34 wherein a programming 
environment of the computer program places restrictions on 
access to the data structure and a value set of the computer 
program during execution thereof. 

55. The system of claim 34 wherein the display engine 
displays a graphical representation of the duplicate copy of 
the data structure. 

56. The system of claim 55 wherein the display engine 
dynamically varies the graphical representation to reflect 
modifications to the duplicate copy of the data structure 
during a run-time of the computer program. 

57. The system of claim 56 wherein the data structure of 
the computer includes classes and objects instantiated from 
the classes, and the display engine displays the graphical 
representation to provide a run-time display of class instan- 
tiations as reflected in the duplicate copy of the data struc- 
ture. 

58. The system of claim 56 wherein the display engine 
displays the graphical representation to provide a run-time 
display of reference variable modifications as reflected in the 
duplicate copy of the data structure. 

59. The system of claim 34 wherein the display engine 
displays a node -link representation of the duplicate copy of 
the data structure. 

60. The system of claim 59 wherein a node of the 
link- node representation represents a class of the data struc- 
ture as reflected in the duplicate copy thereof. 

61. The system of claim 59 wherein a node of the 
link-node representation represents an object of the data 
structure, the object comprising an instantiation of a class of 
the data structure as reflected in the duplicate copy thereof. 
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62. The system of claim 59 wherein a first link of the 
link-node representation represents an object reference. 

63. The system of claim 59 including, responsive to a user 
selection, displaying the link-node representation in a hori- 
zontal orientation. 

64. The system of claim 59 including, responsive to a user 
selection, displaying the link-node representation in a ver- 
tical orientation. 

65. The system of claim 34 wherein the display engine 
displays a zoomed-out view of the data structure, as repre- 
sented by the duplicate copy, in a first graphically distinct 
area and displays a zoom-in view of the data structure, as 
represented by the duplicate copy, in a second graphically 
distinct area. 

66. The system of claim 65 wherein a data structure 
representation presented only in the zoomed-in view pre- 
sents variable and value information. 

67. The system of claim 65 wherein the display engine 
defines a pan view region within the first graphically distinct 
area that is user locatable with reference to the zoomed-out 
view of the data structure, a portion of the duplicate of the 
data structure displayed within the pan view region being 
displayed by the display engine within the second graphi- 
cally distinct area. 

68. The system of claim 65 wherein both the first and 
second graphically distinct areas can be zoomed into and out 
of responsive to a user selection utilizing the display engine. 

69. A system to visualize operation of the computer 
program, the system including: 

a first means for maintaining a duplicate copy of a data 
structure of a computer program during execution of 
the computer program; and 

second means for generating a display of a visual repre- 
sentation of the duplicate copy of the data structure on 
a display device so as to provide a visualization of the 
operation of the computer program. 

70. A machine-readable medium having a sequence of 
instructions stored thereon that, when executed by a 
machine, cause the machine to: 

maintain a duplicate copy of a data structure of a com- 
puter program during execution of the computer pro- 
gram; and 

display a visual representation of the duplicate copy of the 
data structure on a display device so as to provide a 
visualization of the execution of the computer program. 
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UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 



PATENT NO. : 6,429,860 Bl Page 1 of 1 

DATED : August 6, 2002 

INVENTOR(S) : Hughes 



It is certified that error appears in the above- identified patent and that said Letters Patent is 
hereby corrected as shown below: 



Column 1, 

Line 41, delete "JVMs)" and insert - (JVMs) --. 
Column 7, 

Line 6, delete "block 941" and insert -- block 94, --. 
Column 8, 

Line 60, delete "(e.g., jar, zip, etc.)" and insert - (e.g., jar, .zip, etc.) --. 
Column 14, 

Line 47, delete "system of claim 1" and insert -- system of claim 35 --. 



Signed and Sealed this 
Twenty-sixth Day of August, 2003 




JAMES E. ROGAN 
Director of the United States Patent and Trademark Office 
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